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ABSTRACT 

It is widely assumed that developments in information and 
communication technologies are fundamentally transforming and 
improving higher education. As a part of an ongoing evaluation of 
technology-supported pedagogy in one university, our three-year research 
project was designed, on the one hand, to determine if and how selected 
technologies were beneficial for learning and, on the other hand, to offer 
professional development for faculty members. In this paper, we reflect on 
our participation in a pedagogy and technology (referred to as PedTech) 
pilot project, describe some of the relationships that developed between 
ourselves as researchers and evaluators and our faculty collaborators, 
and share what we have learned from this experience. We suggest that 
a scholarship of teaching approach to evaluating innovations in teaching 
and learning is one way to support institution-wide adoption. 
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RESUME 

Presumons que les developpements dans les technologies de 
1’ information et de la communication transforment et ameliorent 
1’ education postsecondaire. Par une evaluation continue de technologies 
pedagogiques dans une universite, notre projet de recherche avait deux 
objectifs de base. Le premier objectif etait de determiner si les technologies 
choisies amelioreraient I’apprentissage. Le deuxieme etait d’offrir le 
developpement professionnel aux professeurs cibles. Dans cet article, 
nous reflechissons sur notre participation dans ce projet pilote de nature 
techno-pedagogique. Nous decrivons les relations developpees entre nous, 
les chercheurs-evaluateurs, et les professeurs collaborateurs, pour partager 
les legons et les pratiques de reussite. Nous croyons qu’une approche 
institutionnelle d’evaluation des technologies pedagogiques innovatrices 
est un moyen d’etendre cette application. 


INTRODUCTION 

Universities in Canada have the responsibility to meet society’s needs 
by preparing citizens for a rapidly changing, knowledge-based world. It 
is widely assumed that developments in information and communication 
technologies are fundamentally transforming and improving higher 
education (Advisory Committee for Online Learning, 2001). To address 
the challenges posed by the changes, many universities have implemented 
pilot projects to test the effectiveness of new technologies for learning. 
These pilot projects usually involve providing training and support for 
faculty members’ innovations in teaching with technology. As a part of 
an ongoing evaluation of technology-supported pedagogy in classrooms 
at one university, our research was designed to determine if and how 
selected technologies were beneficial for learning, and to offer professional 
development for faculty members. In this article, we reflect on our 
participation in an institution-wide pedagogy and technology (referred 
to as PedTech) pilot project, describe some of the relationships that 
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developed between ourselves as researchers and evaluators and our faculty 
collaborators, and share what we have learned from this experience. 

BACKGROUND AND THEORETICAL FRAMEWORK 

In 1999, our university received external funding for a three-year 
university-wide pilot projeet aimed at supporting the speeifie elassroom- 
based PedTech projects of individual faculty members. The goal was 
for faculty to transform teaching and learning in their courses through 
pedagogically-sound applications of various computer communication 
technologies. It was expected that these innovations would lead to 
institutional-level adoption of technology. As members of an on-campus 
research center, we were involved in two aspects of the pilot project: 
(a) evaluating the effectiveness of teaching and learning strategies 
using technology, and (b) providing training and support to faculty. Our 
involvement took the form of designing and implementing the evaluations 
of specific innovations and helping faculty to learn about evaluation as a 
tool for ongoing teaching improvement. 

In our work with faculty we were influenced by theoretical frameworks 
in teaching and learning, instructional design, technology implementation 
and educational research. For example with respect to teaching and 
learning, we drew from constructivist theories, which emphasize strategies 
that promote cognitive engagement and meaningful involvement with 
tasks. Active learning — interaction with teachers, classmates, and course 
materials — allows learners to construct their own meaning (Jonassen, 
1999), preferably within authentic learning situations (Brown, Collins, & 
Duguid, 1989). The implication of this perspective for instruction is that 
teachers become guides or coaches who provide learners with appropriate 
scaffolding (Vygotsky, 1978) so that students can maximize their ability 
to apply their newly acquired knowledge in personally meaningful 
contexts. There is a pervasive view in the higher education literature that 
technology can effectively support constructivist learning environments 
(e.g., Hannafln & Land, 1997; Harasim, 1999; Hiltz, 1994; Rogers, 
2000; Twigg, 2000). Research has recognized the value of approaching 
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instructional design through a number of learning-centered principles that 
focus on cognitive, metacognitive, motivational and affective factors, 
and individual differences (Lambert & McCombs, 1998). Good practice 
in undergraduate education involves interaction and active learning, 
is organized to support a range of learning preferences (Chickering & 
Gamson, 1987), and includes a variety of instructional materials (Brown, 
2000). For technology to play an effective role in this process, it must be 
wisely integrated with pedagogy and institutional infrastructure (Bates & 
Poole, 2003; Daniel, 1996; Richey, 1997). This means that technologies 
should be chosen appropriately, based on their features (Laurillard, 2002) 
and their use should be designed to support specific learning outcomes 
(Gandell, Weston, Finkelstein & Winer, 2000; Sharpe & Bailey, 1999). 

We approached the PedTech evaluations from an educational research 
perspective that was grounded in quasi-experimental studies using 
quantitative methods (Campbell & Stanley, 1963; Gall, Borg & Gall, 
1996). We were also influenced by our colleagues in the research center, 
who had an established reputation for quantitative meta-analyses, (e.g., 
Azevedo & Bernard, 1995; Lou, Abrami & d’Appollonia, 2001). 

THE PILOT PROJECT 

The pilot project was originally envisaged as centrally coordinated, 
with pedagogical, technical, and research expertise provided separately by 
three on-campus support centers. These included (a) teaching and learning 
services, (b) instructional technology services, and (c) the research center 
with which we were affiliated. We were expected to design research studies 
in keeping with our traditions that would be acceptable for publication 
in peer-reviewed education journals. On the other hand, we were also 
expected to collaborate with faculty in designing the evaluations of their 
technology-enhanced learning environments. This implied that we had a 
mandate for both research and evaluation. 

Eighteen months into the pilot project, it became evident that a radical 
change in focus had occurred. Each Eaculty within our university was 
at a very different stage of development with respect to implementing 
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technology: some had established well-staffed centers to support PedTech 
development, while others had no experience and no infrastructure. Many 
individual professors were reluctant to change their teaching practices — 
this reduced the pool of potential collaborators. The group of willing 
associates was further limited by the fact that many other professors were 
uncomfortable with the notion of research in their classrooms. They seemed 
particularly concerned by the possibility of evaluation by educational 
researchers of their experiments with using technologies for teaching 
and learning. In light of profound objections to the centralized model, we 
designed an alternate approach to studying the innovations funded by the 
grant. We proposed two types of research activities (referred to as Tier 1 
and Tier 2) with which faculty could choose to become involved, and a 
third institutional-level approach (Tier 3) which we and other members of 
the research center undertook independently. This proposal was generally 
accepted by the pilot project’s steering committee and formed the structure 
of the final project. 

In Tier 1 activities we concentrated on the evaluation — both formative 
and summative — of each of the funded PedTech interventions, with a 
focus on student learning and motivation, as well as whether or not the 
technology worked as expected. The extent of each faculty member’s 
involvement in this process depended on their interest in the results and 
their willingness to collaborate. The aim of this evaluative process was 
to provide timely, constructive feedback over the course of one semester 
to improve and extend the interventions (or indeed change or discontinue 
their use as the data suggested). Our role in this case was advisory: first 
to find or design evaluation instruments, and second to clarify for the 
participating faculty members and discuss with them the implications of 
the data collected. 

Tier 2 research involved a relatively small number of in-depth 
investigations in areas of instructional concern to associated faculty 
members. We identified the research questions and formulated the 
research designs in collaboration with them. Our role here was negotiated 
individually, but involved working with faculty to design and implement 
a more traditional research study of a PedTech innovation. Typically the 
collaboration lasted more than one semester. 
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In Tier 3 research we explored areas of interest to ourselves or that we 
felt would make the kind of larger scale impact on the university that the 
pilot project was originally intended to have. We initiated investigations 
that were broader in scope, and that transcended the particular interests 
of one or a few faculty members. We had hopes that the results of Tier 3 
initiatives would be directly useful for teaching and learning at a pan- 
university level. With this in mind, we planned to disseminate our results 
as broadly as possible. 

A sample listing of the studies conducted in the pilot project is available 
in Table 1. We elaborate below on one study at each level to illustrate the 
relationships that evolved with faculty members as their studies unfolded. 

Tier 1 Example 

Tier 1 activities generally evaluated computer-based interventions 
designed to enhance the acquisition of specific learning outcomes. In this 
example, an instructor using a graphical authoring software (Inspiration®) 
for presentation purposes was interested in evaluating electronic concept 
mapping as a learning tool in her subject area. She wanted to know how 
the technology could be used in her classes to improve learning. To acquire 
this information, we undertook a full formative evaluation by collecting 
data prior to, during, and following the use of the tool in the class. 

Our relationship with this faculty member was most often one of 
gathering information, summarizing, and synthesizing the data so that 
we could all understand its implications. Then she could apply these new 
insights in her teaching. In response to her need for instruction in the use 
of the tool before she could proceed with the implementation, we trained 
her and the students. When she required technical support we facilitated 
it. The third author was one of the researchers who worked closely with 
this instructor, and her reflections illustrate that the instructor seemed to 
enjoy brainstorming about conducting evaluations of her teaching with 
technology and working together on pedagogical solutions. 

In retrospect, she seemed to be very comfortable with the larger 
team and with me. I believed that my major role in working with 
this faculty member was to assist her in using the tool in her 
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context. I hoped to help construct an evaluation design that would 
be of minimal interference to her teaching and students’ learning, 
yet suitably sophisticated to yield meaningful and valid data. As 
well, my role was to fuel her already existing enthusiasm — she was 
making a significant change in her teaching so that her students 
could learn the discipline, her discipline, better. More importantly, 
she was deliberately looking to me for ways to help her students 
understand content that was very difficult for them and she knew that 
her past attempts at trying to guide them had not been as successful 
as she had hoped. However, she chose to stay within the parameters 
of Tier 1 because she did not have release time to spend developing 
modified materials and testing them. In the end, she was satisfied 
that the tool helped her students learn better. 

This time constraint was typical of most faculty members involved in 
Tier 1 projects. 

Tier 2 Example 

In Tier 2 research, faculty members embraced an in-depth study that 
was of key interest to them, and typically, we assisted them in designing, 
developing, investigating, and reporting the results of their research 
questions. Our relationship with faculty members in Tier 2 projects 
developed into extended partnerships. Although faculty members generated 
the initial ideas, we contributed our classroom-based research experience 
and assisted them with implementation. In these projects, we were invited 
to share our expertise in educational research and instructional design 
in support of investigations that were well-grounded in the educational 
research tradition. 

One faculty member wished to utilize technology (digital animation) 
to enhance critical thinking. The first and third authors met with the faculty 
member regularly, observing her in the classroom to become acquainted 
with her teaching styles, as well as with the technology she wanted to 
use and the questions she wanted to address. We undertook a literature 
search on critical thinking on her behalf, and met frequently as a research 
team with the faculty member as an active participant, examining together 
strategies for teaching critical thinking in her discipline. 
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We also assisted her with the design of assessments of students’ 
learning. We arranged for a doctoral student with expertise in critical 
thinking protocols to analyze students’ assignments for evidence of 
critical thinking. We then summarized and helped the instructor interpret 
the findings so she eould reeoneeptualize her eourse for the future and 
modify her instructional strategies where necessary. This partnership has 
continued through three more academic years until the point where the 
professor now feels her course has reached a level of effectiveness that 
satisfies her. The first author has been working with her throughout the 
project’s life and is currently helping to write a reflective article on the 
process for publication in a journal devoted to excellence in teaching and 
learning at the postsecondary level. Her reflections focus on the longer- 
term impact of the pilot project. 

One of the interesting aspects of this project is that the availability 
of seed funding for technological interventions has inspired a course 
redesign effort that has lasted several terms. We’ve seen significant 
improvements in student satisfaction over time, and, in addition, 
the professor has learned so much about using varied instructional 
strategies for improved learning of difficult concepts. Her confidence 
with using technology appropriately has also been enhanced. 

Tier 3 Examples 

By exploring the systemic impact of technology implementation, we 
intended to identify variables influencing institutional transformation. Tier 
3 activities included two different institution-wide surveys of classroom- 
based technology use and a comprehensive and synthetic review of the 
literature on technology integration. These studies did not depend upon 
establishing relationships between ourselves and individual faculty 
members. Rather, Tier 3 research was concerned with examining issues and 
concerns that affected the broader university community. Tier 3 research 
had the potential to speak to a large audience, transcending the particular 
context of specific university classrooms. Conducting it made no demands 
on individual faculty members. Tier 3 comprised three research studies, 
described as follows. 
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1. Technology Integration and Students With Disabilities. The second 
author was involved in an institution- wide project initiated with the office 
of services for disabled students. The study examined how the integration 
of technology within the various Faculties impacted on students with 
disabilities, who represent about 10% of the total student population at our 
university. As computer-mediated technologies are becoming increasingly 
incorporated into students’ university experience, there is a growing concern 
(e.g., Burgstahler, 2002; Fichten, Asuncion, Barile, Genereux, Fossey, 
Judd, Robillard, De Simone, & Wells, 2001; Gibson, 2002) that students 
with disabilities may have difficulty accessing or using these technologies 
to their advantage. This team also developed a questionnaire for professors 
on their use of technology, especially whether and how they had adapted 
the technology to meet the needs of their students with disabilities. The 
second author described the process and findings as follows. 

Working with the Coordinator for Services for Disabled Students, I 
collaborated on the development of a questionnaire for students with 
disabilities, asking them about their experience with technology 
in their classes, labs, library, and at home. Services for Disabled 
Students coordinated the preparation, distribution, and collection of 
the student questionnaires. Together we analyzed and interpreted 
the findings. We found that while professors did not take students 
with disabilities into account when implementing technology, they 
were more than willing to be trained, on a just-in-time basis, to 
adapt their use of technology in the classroom. 

Results were compiled and analyzed for public presentations 
(Bissonnette, Schmid, & McWhaw, 2002; McWhaw, 2002) on the needs 
and experiences of the faculty. Our university now has the data to make 
informed decisions about how to ensure access to technology-based 
teaching and learning tools for disabled students. Improved institutional 
policies and procedures, as well as approaches for faculty development 
in designing or adapting technology-based courses for students with 
disabilities, will also benefit other university communities with respect to 
accessibility to technologies that meet particular learning needs. 

2. Student Perceived Effectiveness of Technology Use. In a second 
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project, other researchers within our center conducted a university-wide 
survey of student perspectives of the effectiveness of using computer 
technologies for learning. Some of the critical questions addressed by 
the study were (a) what role does technology play in student perceived 
effectiveness of a class; (b) what is the relationship between technology 
use, how that technology is used, and student perceived effectiveness of 
the class; and (c) is the use of technology seen by students as an essential 
component of learning? The variables under investigation included degree 
of learner control, specific and general benefits to students, previous 
experience, technical support, access and demographics. The results 
suggested a positive relationship between global course evaluations and 
the learning experiences that students engaged in. Students also indicated 
that they valued the use of computer technologies for learning. (Lowerison, 
Sclater, Schmid & Abrami, 2003). 

The next year, the same group of researchers conducted a follow- 
up study which investigated the role that computer technology plays in 
transforming the learning process in higher education. Results suggested a 
relationship between computer technology, active learning, and perceived 
course effectiveness. Students who use computer technology a lot appear 
to benefit the most from active learning (Lowerison et. al., 2004). 

3. Successful Practices. In the third Tier 3 study, the first and third 
authors conducted a comprehensive review of recent (1995-2001) literature 
on technology integration in higher education. We aimed to identify and 
understand successful classroom-based pedagogical and technological 
practices, in order to share them with other faculty members. We planned 
to extract the lessons learned from other studies as a basis for encouraging 
faculty members to adapt the ideas to their own contexts, creating their 
own discoveries, and, in turn, sharing them with other colleagues to 
eventually construct communities of practice in technology-supported 
teaching and learning. The outcome of these learning communities would 
be to offer faculty a virtual space, an arena where they could engage in 
exploration and creative pursuits of technology-supported teaching and 
learning practices. The key findings of this synthesis are described below. 
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We found that successful teaching and learning practices using 
technology in the postsecondary classroom could be categorized into 
one of: (a) technical practices, (b) support and training practices, and (c) 
instructional practices. Technical practices (e.g., Mitra, Steffensmeier, 
Lenzmeier & Massoni, 1999; Oliva & Pollastrini, 1995) include reliable, 
universal access to appropriate hard and software, with regular equipment 
upgrades. Standardization is highly recommended. Support and training 
practices (e.g., Gibson & Nocente, 1998; Sprague, Kopfman, & deLevante, 
1998; Strudler, McKinney, & Jones, 1995; Wager, Heye, & Tsai, 1995) 
include both pedagogical and technical training offered by competent and 
available instructional and IT specialists. As well, institutional planning 
efforts are essential. Instructional practices (e.g., Carlson & Gooden, 1999; 
Halpin, 1999; Peters, O’Brien, Briscoe, & Korth, 1995; Siegel, Good, & 
Moore, 1996; Smith, 2000) include constructivist teaching strategies, 
modeling, group work, practice activities and the choice of appropriate 
software. We found it interesting that many research reports purporting to 
describe evaluations of the effectiveness of technology-enhanced teaching 
and learning focused on administrative rather than on teaching and learning 
issues, and note here that two of the three categories we identified in the 
literature were not directly related to teaching activities. Few of the studies 
reported learning gains related to the use of technology. 

The successful practices review raised our awareness of other 
postsecondary initiatives in technology integration. Conducted at the same 
time as Tier 1 and Tier 2 projects, we found descriptions in the literature 
of many of the challenges we were facing. The more we read, the more we 
came to understand that we were not alone in our efforts to help professors 
use technology effectively. 


DISCUSSION 

After three years on the various PedTech evaluation and research 
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projects, we felt that our work was of value to the university in encouraging 
a culture of evaluation among some professors. Although relatively few 
faculty members identified projects at either level, there were some who 
viewed the idea of Tier 1 evaluation or Tier 2 research with enthusiasm, 
considering it an opportunity for them to investigate their technology-based 
instructional strategies. Our participating professors fit into this group. We 
know that they appreciated the opportunity to explore the impact of their 
use of technologies on their students. They tended to be confident about 
their teaching, but were looking for ways to better support their students’ 
learning. They were also comfortable with the process of evaluation. 

There was, however, substantial resistance among professors generally 
to getting involved in either Tier 1 or Tier 2 projects. We speculate now 
that there were a number of reasons for this resistance and our reading of 
the successful practices literature has helped us make sense of it. Some 
professors may not have understood the potential of evaluation to improve 
not only teaching but also student learning. They may also have been 
unclear about our mandate or roles (see Cross, 2000; Gelzheiser & Meyers, 
1996). The discomfort may have been exacerbated by the suggestion that 
researchers from within an educational research center should conduct 
evaluations in other disciplines. Some professors seemed to be concerned 
about having their performance as instructors assessed by outsiders. In 
a culture where teaching quality contributes to decisions on promotion 
and tenure, this lack of confidence is understandable, but it misses the 
point that building formative evaluation into teaching innovation is a tool 
for improvement that can save time and resources in the long run. Other 
professors seemed to view assessment of their teaching as contrary to their 
academic freedom (Gray, 1997). 

According to Wiggins (2000), this uneasiness with evaluation — even 
formative evaluation — is fostered because it is perceived as something 
punitive, rather than as a tool for improving teaching and learning. This 
may be especially true in large-scale projects such as ours with centralized 
or administrative control. In some cases, we were not able to carry out 
our evaluation mandate. It became the responsibility of individual grant 
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recipients to conduct and report their own evaluations. This development 
highlighted for us the tendency of institutional politics, interdepartmental 
competitiveness, and a “technology push” attitude to complicate efforts 
to improve teaching and learning. We also noted the lack of value 
accorded educational research in the postsecondary environment. As 
well, we observed the importance of clarifying the responsibilities of 
various organizational units up front, at the beginning of an institution- 
wide pilot project. 

The Tier 3 studies were conceived to answer some of the broader 
questions we were faeed with in the pilot project, specifically related to 
the organizational weaknesses in planning for technology implementation 
that are widely reported in the literature and which we experienced as 
well. Barriers to the adoption of innovations are well documented, as are 
the factors involved in the resistant behaviours of faculty and students 
(e.g.,Anderson,Varnhagen,& Campbell, 1998;Ertmer, 1999; Rogers, 1999; 
Fabry & Higgs, 1997; Schuell & Farber, 2001). Through the identification 
of successful practices we have been able to bring the results of other 
studies to our understanding of what went on in this project, including the 
larger questions related to policy and planning for technology integration. 

On reflection, and now that the project is over, we realize that we 
did not explicitly draw on theoretical frameworks in evaluation, faculty 
development, or technology integration. Though our practice as educational 
researchers had provided experience with the tools and methods we would 
use to design studies, collect and analyze data, we were not completely 
prepared for the challenges we would face. In retrospect, we feel we 
were not familiar enough with the politics of evaluation. Fevin-Rosalis 
(2003) points out that research and evaluation are separate disciplines with 
different purposes, and that though the roles of researchers and evaluators 
overlap, they are distinct. The purpose of research is to acquire knowledge, 
while evaluation is a tool for decision makers. In our context, evaluations 
of PedTech innovations were to help inform the university about where 
to invest institutional resources for technology in the future. As such 
we were searching for evidence about how to implement technology in 
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ways that supported enhanced student learning. To do this we needed 
faculty members who were willing to examine their practice using Tier 1 
approaches at least. Even Tier 2 projects were limited in scope and impact 
to the individual professors’ personal circle of colleagues and influence. 
We would not be able to generalize our results to the broader university 
community. Institutional change, an important goal of the PedTech 
Project, depends on broadly applicable results, rather than individual 
accomplishments in a few specific courses. 

We do not feel that our research mandate was compromised by the 
reluctance of professors to participate in the pilot project evaluations, 
however. The Tier 3 projects all contributed to a broader understanding of 
technology use in postsecondary teaching and learning and we have begun 
to successfully disseminate this new knowledge. 

One area that troubled us was that our obligations as evaluators seemed 
to be in conflict with our faculty development goals. Ramsden (1992) 
points out that evaluating teaching effectiveness is best done by instructors 
rather than to them. In Tier 1, although we offered to collaborate with 
professors, we were frequently perceived as external auditors who would 
report back to the university leadership on what they were doing “wrong.” 
The Tier 2 example is for us an excellent example, not only of thoughtful 
use of technology and the ongoing nature of course design, but also of 
the value of a scholarship of teaching (Boyer, 1990; Cambridge, 1999; 
Hakim, 2002) and action research (Carson & Sumara, 1 997) approach to 
improving instructional practice over time. We interpret the continuing 
relationship with this professor as evidence that research can help improve 
teaching practice and student learning as well. And we appreciate the 
additional perspective on faculty development (Weimer, 2002) as an 
ongoing activity that this study provides. 

In terms of technology integration frameworks, we were aware that 
a major hurdle in institution-wide implementations is that they require 
simultaneous changes in vision, curriculum, instruction, and assessment 
practices. Such changes can be uncomfortable for academics, and they tend 
to occur at different rates for different groups and individuals (Anderson, 
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Vamhagen & Campbell, 1998; Ertmer, 1999; Powers, Anderson, & Love, 
2000; Rogers, 1999). We were not in a position to affect university policy 
in, for example, reward and recognition of professors’ initiatives with 
technology. Nor were we able to help our collaborators find the time in 
their busy schedules to design evaluation instruments or analyze data. 
Although we were concerned about the potential for an over-emphasis on 
technology and an under-emphasis on the pedagogical aspects of the pilot 
project (Cognition and Technology Group at Vanderbilt, 1996), we were 
not able to address this except with individual faculty members. 

REFLECTIONS 

There is some evidence in the evaluations we conducted and in instructor 
perceptions that student learning of difficult concepts was enhanced in the 
Tier 1 and Tier 2 projects. We cannot tell whether or not the technology 
alone was responsible, but we believe that the process of evaluation had 
an impact on the teaching, which led to improved student learning. One 
important question for universities to address is how to evaluate the direct 
effects of technology on learning as separate from course design and/or 
teaching approaches. We wonder whether this is even possible or desired. 
What methodologies, for example, are appropriate in evaluation? We 
found that multiple methods of data collection and analysis gave us more 
confidence in our results than a strictly quasi-experimental approach. 

Our experience confirms the value of grants for getting professors 
involved in PedTech innovations as they allow for experimentation with 
new technologies and teaching methods. We see these seed grants as the 
first step to setting up a system for ongoing teaching improvement in 
universities. However, the expectation that university-wide technology 
adoption would follow from a few individual implementations was 
unrealistic. Technology integration needs more than seed funding to be 
institutionalized (Ely, 1997). 

Our question, and the question that comes from the literature review, 
is how to put systems in place in universities to support widespread 
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technology implementation and pedagogical improvement. Although we 
have some idea of what is needed for professors to be motivated to adopt 
unfamiliar learning technologies (e.g., rewards, recognition, support, 
financing, and more time), we wonder how universities can reallocate their 
resources to sustain a community of academics who value technology as a 
tool for improving teaching and learning. Some of the resources that need 
to be provided include technical expertise and infrastructure, training in 
technology and pedagogy, and the pedagogical consultants to offer ongoing 
support. Decisions about the nature of this support (e.g., centralized or 
decentralized structures) need to take into account the particular culture of 
each institution (see Ives, 2002). 

The question that underlies all of the above is how universities can 
promote and build a culture for evaluation as a tool for improving teaching. 
Our experience in this project suggests that a scholarship of teaching 
approach is one way to do this. This approach is based on the sharing of 
individual teaching experiences with colleagues, being open to receiving 
critiques and suggestions for improvement, and building upon the lessons 
about learning offered by others in a systematie way. Sinee we modeled 
this behaviour in our relationships with our participating faculty members, 
we recognize the need for system-wide encouragement in order to expand 
the community beyond the few professors with whom we worked. We 
think that this is particularly important in the area of technology because 
determining appropriate choices is such a complex matter. Other people’s 
experiences, both failures and successes, are helpful in this regard. 

We also note that there are special skills needed by pedagogical 
consultants and faculty developers to make it possible for them to work 
with large numbers of faculty from many different disciplines. These skills 
include excellent interpersonal communication and conceptual flexibility 
in addition to evaluation, research, and technical skills. An additional skill 
is the critically important ability to reflect upon one’s experiences and to 
apply the lessons learned from them in future practice (Schon, 1987). 

It has taken us a few years after the completion of the pilot project 
to be able to reflect upon our experiences. Each of us has moved on to a 
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new role — in faculty development, program evaluation, or teaching and 
research — in a different institution. Our new environments have provided 
us with the frameworks (e.g., Cambridge, 1999; Chism, 2004; Levin- 
Rosalis, 2003; Ramsden, 1992; Saroyan & Amundsen, 2004; Weimer, 
2002) we laeked when we first beeame involved in the PedTech projects. 
This has given us new language and new understanding of our roles and 
our contributions. As we face similar challenges now, we can draw on the 
lessons we learned. We share these experiences in the spirit of ongoing 
improvement of postsecondary teaching and learning with technology. Z 
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